TM-LDA: Efficient Online Modeling of the Latent Topic Transitions in Social Media

Latent topic analysis has emerged as one of the most effective methods for classifying, clustering and retrieving textual data. However, existing models such as Latent Dirichlet Allocation (LDA) were developed for static corpora of relatively large documents. In contrast, much of the textual content on the web, and especially social media, is temporally sequenced, and comes in short fragments such as Tweets, Facebook status updates, or comments on YouTube. In this paper we propose a novel topic model, Temporal-LDA or TM-LDA, for efficiently mining streams of social text such as a Twitter stream for an author, by modeling the topics and topic transitions that naturally arise in such data. TM-LDA learns the transition parameters among topics by minimizing the prediction error on topic distribution in subsequent postings. After training, TM-LDA is thus able to accurately predict the expected topic distribution in future posts. To make these predictions more efficient for a realistic online prediction setting, we develop an efficient updating algorithm to adjust transition parameters, as new documents stream in. Our empirical results, over a corpus of over 30 million Twitter posts show that TM-LDA significantly outperforms state-of-the-art static LDA models for estimating the topic distribution of new documents over time. We also demonstrate how TM-LDA is able to highlight interesting variations of common patterns of behavior across different cities, such as differences in the work-life rhythm of cities, and factors responsible for area-specific problems and complaints.